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The  main  results  obtained  and  published  during  the  period  covered  by  this  report,  August  1988 
-  July  1990,  are  described  below  together  with  references  given  to  the  corresponding 
publication.  •— 

1 .  The  Interacting  Multiple  Model  Algorithm  for  Systems  with  Markovian  Switching  Coefficients, 
(Henk  A.  Blom  and  Yaakov  Bar-Shalom,  IEEE  Transactions  on  Automatic  Control  Vol.  33, 
No.  8,  August  1988) 

An  important  problem  in  filtering  for  linear  systems  with  Markovian  switching  coefficients 
(dynamic  multiple  model  systems)  is  the  one  of  management  of  hypotheses,  which  is  necessary 
to  limit  the  computational  requirements.  A  novel  approach  to  hypotheses  merging  has  been 
developed  for  this  problem.  The  novelty  lies  in  the  timing  of  hypotheses  merging.  When 
applied  to  the  problem  of  filtering  for  a  linear  system  with  Markovian  coefficients  this  yields  an 
elegant  way  to  derive  the  interacting  multiple  model  (IMM)  algorithm.  Evaluation  of  the  IMM 
algorithm  makes  it  clear  that  it  performs  very  well  at  a  relatively  low  computational  load.  These 
results  imply  a  significant  change  in  the  state  of  the  an  of  approximate  Bayesian  filtering  for 
systems  with  Markovian  coefficients. 

2.  Failure  Detection  Via  Recursive  Estimation  for  a  Class  of  Semi-Markov  Switching  Systems, 
(L.  Campo,  P.  Mookeijee  and  Y.  Bar-Shalom,  Proceedings  1988  IEEE  CPC.  Austin,  Texas) 

An  area  of  current  interest  is  the  estimation  of  the  state  of  discrete-time  stochastic  systems  with 
parameters  which  may  switch  among  a  finite  set  of  values.  The  parameter  switching  process  of 
interest  is  modeled  by  a  class  of  semi-Markov  chains.  This  class  of  processes  is  useful  in  that 
it  pertains  to  many  areas  of  interests  such  as  the  failure  detection  problem,  the  target  tracking 
problem,  socio-economic  problems  and  in  the  problem  of  approximating  nonlinear  systems  by 
a  set  of  linearized  models.  It  is  shown  in  this  paper  how  the  transition  probabilities,  which 
govern  the  model  switching  at  each  time  step,  can  be  inferred  via  the  evaluation  of  the 
conditional  distribution  of  the  sojourn  time.  Following  this,  a  recursive  state  estimation 
algorithm  for  dynamic  systems  with  noisy  observations  and  changing  structures,  which  uses 
the  conditional  sojourn  time  distribution,  is  derived  and  and  applied  to  a  failure  detection 
problem. 
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3.  Distributed  Adaptive  Estimation  with  Probabilistic  Data  Association,  (K.C.  Chang  and  Y. 
Bar-Shalom,  Automatica.  Vol.  25,  No.  3,  pp.  359-369, 1989) 

The  probabilistic  data  association  filter  (PDAF)  estimates  the  state  of  a  target  in  a  cluttered 
environment.  This  suboptimal  Bayesian  approach  assumes  that  the  exact  target  and 
measurement  models  are  known.  However,  in  most  practical  applications,  there  are  difficulties 
in  obtaining  an  exact  mathematical  model  of  the  physical  process.  In  this  paper,  the  problem  of 
estimating  target  states  with  uncertain  measurement  origins  and  underlain  system  models  in  a 
distributed  manner  is  considered.  First,  a  scheme  is  described  for  local  processing,  then  the 
fusion  algorithm  which  combines  the  local  processed  results  into  a  global  one  is  derived.  The 
algorithm  can  be  applied  for  tracking  a  maneuvering  target  in  a  cluttered  and  low  detection 
environment  with  a  distributed  sensor  network. 

4.  An  Adaptive  Dual  Controller  for  a  MIMO-ARMA  System,  (P.  Mookerjee  and  Y.  Bar-Shalom, 
IEEE  Transactions  on  Automatic  Control.  Vol.  34,  No.  7,  July  1989) 

An  explicit  adaptive  dual  controller  has  been  derived  for  a  multiinput  multioutput  ARMA 
system.  The  plant  has  constant  but  unknown  parameters.  The  cautious  controller  with  a 
one-step  horizon  and  a  new  dual  controller  with  a  two-step  horizon  are  examined.  In  many 
instances,  the  myopic  cautious  controller  is  seen  to  turn  off  and  converges  very  slowly.  The 
dual  controller  modifies  the  cautious  control  design  by  numerator  and  denominator  correction 
terms  which  depend  upon  the  sensitivity  functions  of  the  expected  future  cost  and  avoids  the 
turn-off  and  slow  convergence.  Monte-Carlo  comparisons  based  on  parametric  and 
nonparametric  statistical  analysis  indicate  the  superiority  of  the  dual  controller  over  the  cautious 
controller. 
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5.  Time- Reversion  of  a  Hybrid  State  Stochastic  Difference  System,  (Henk  A.P.  Blom  and 
Yaakov  Bar-Shalom,  Proc.  1989  IEEE  Intn'l.  Conf.  on  Control  &  Applications.  Jerusalem, 
Israel,  April  1989,  and  in  IEEE  Trans.  Info.  Theory.  IT-36,  July,  1990) 

This  paper  develops  the  reversion  in  time  of  a  stochastic  difference  equation  in  a  hybrid  space, 
with  a  Markovian  solution.  The  reversion  is  obtained  by  a  martingale  approach,  which 
previously  led  to  reverse  time  forms  for  stochastic  equations  with  Gauss-Markov  or  diffusion 
solutions.  The  reverse  time  equations  follow  from  a  particular  non-canonical  martingale 
decomposition,  while  the  reverse  time  equations  for  Gauss-Markov  and  diffusion  solutions 
followed  from  the  canonical  martingale  decomposition.  The  need  for  the  non-canonical 
decomposition  stems  from  the  hybrid  state  space  situation.  The  non-Gaussian  discrete  time 
situation  leads  to  reverse  time  equations  that  incorporate  a  Bayesian  estimation  step. 

6.  A  New  Controller  for  Discrete-Time  Stochastic  Systems  with  Markovian  Jump  Parameters,  (L. 
Campo  and  Y.  Bar-Shalom,  11th  IFAC  World  Congress.  Tallinn,  USSR,  Aug.  1990 

A  realistic  stochastic  control  problem  for  hybrid  systems  with  Markovian  jump  parameters  may 
have  the  switching  parameters  in  both  the  state  and  measurement  equations.  Furthermore,  both 
the  system  state  and  the  jump  states  may  not  be  perfectly  observed.  Prior  to  this  work  the  only 
existing  implementable  controller  for  this  problem  was  based  upon  a  heuristic  multiple  model 
partitioning  (MMP)  and  hypothesis  pruning.  In  this  paper  a  stochastic  control  algorithm  for 
stochastic  systems  with  Markovian  jump  parameters  was  developed.  The  control  algorithm  is 
derived  through  the  use  of  stochastic  dynamic  progamming  and  is  designed  to  be  used  for 
realistic  stochastic  control  problems,  i.e.,  with  noisy  state  obeservations.  The  state  estimation 
and  model  identification  is  done  via  the  recently  developed  Interacting  Multiple  Model 
algorithm.  Simulation  results  show  that  a  substantial  reduction  in  cost  can  be  obtained  by  this 
new  control  algorithm  over  the  MMP  scheme. 
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7.  From  Piecewise  Deterministic  To  Piecewise  Diffusion  Markov  Processes,  (Henk  A.P.  Blom, 
Proc.  IEEE  CPC  19881 

Piecewise  Deterministic  (PD)  Markov  processes  form  a  remarkable  class  of  hybrid  state 
processes  because,  in  contrast  to  most  other  hybrid  state  processes,  they  include  a  jump 
reflecting  boundary  and  exclude  diffusion.  As  such,  they  cover  a  wide  variety  of  impulsively 
or  singularly  controlled  non-diffusion  processes.  Because  PD  processes  are  defined  in  a 
pathwise  way,  they  provide  a  framework  to  study  the  control  of  non-diffusion  processes  along 
same  lines  as  that  of  diffusions.  An  important  generalization  is  to  include  diffusion  in  PD 
processes,  but,  as  pointed  out  by  Davis,  combining  diffusion  with  a  jump  reflecting  boundary 
seems  not  possible  within  the  present  definition  of  PD  processes.  This  paper  presents  PD 
processes  as  pathwise  unique  solutions  of  an  Ito  stochastic  differential  equation  (SDE),  driven 
by  a  Poisson  random  measure.  Since  such  an  SDE  permits  the  inclusion  of  diffusion,  this 
approach  leads  to  a  large  variety  of  piecewise  diffusion  Markov  processes,  represented  by 
pathwise  unique  SDE  solutions. 

8.  Control  of  Discrete-Time  Hybrid  Stochastic  Systems  (L.  Campo  and  Y.  Bar-Shalom,  Proc. 
1990  ACC.  May  1990;  to  appear  in  IEEE  Trans.  Auto.  Control.  1991) 

ft 

A  realistic  stochastic  control  problem  for  hybrid  systems  with  Markovian  jump  parameters  can 
have  the  switching  parameters  in  both  the  state  and  measurement  equations.  Furthermore,  both 
the  system  state  and  the  jump  states  are,  in  general,  not  perfectly  observed.  Currently  there  are 
only  two  existing  controllers  for  this  problem.  One  is  based  upon  a  heuristic  multiple  model 
partitioning  (MMP)  and  hypothesis  pruning.  The  other  utilizes  the  entire  future  tree  of  models, 
and  is  called  the  Full-Tree  (FT)  controller.  The  performance  of  the  latter  is  superior  to  the 
former  and  their  complexities  are  similar.  In  this  paper  we  present  a  new  stochastic  control 
algorithm  for  stochastic  systems  with  Markovian  jump  parameters.  This  control  algorithm  is 
derived  through  the  use  of  stochastic  dynamic  programming  and  is  designed  to  be  used  for 
realistic  stochastic  control  problems,  i.e.,  with  noisy  state  observations.  This  new  scheme, 
which  is  based  upon  the  interaction  of  r  (the  number  of  models)  model-conditioned  Riccati 
equations,  has  a  natural  parallelism  and  is  straightforward  to  implement.  The  state  estimation 
and  model  identification  is  done  via  the  recently  developed  Interacting  Multiple  Model 
algorithm.  Simulation  results  show  that  a  substantial  reduction  in  cost  can  be  obtained  by  this 
new  control  algorithm  over  the  MMP  scheme.  Furthermore,  the  performance  of  the  new 
algorithm  is  shown  to  be  practically  the  same  as  that  of  the  FT  scheme  even  though  the  new 
scheme,  which  has  a  fixed  amount  of  computations  at  each  step  of  the  recursion,  is  much 
simpler  to  implement  than  both  the  MMP  and  FT  algorithms. 
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9.  Discrete  Time  Point  Process  Filter  for  Image  Based  Target  Mode  Estimation  (C.  Y ang  and  Y. 
Bar-Shalom,  Proc.  1990  IEEE  CPC:  to  appear  in  IEEE  Trans.  Auto.  Control.  1991) 

The  performance  of  tracking  and  prediction  systems  of  a  maneuvering  target  can  be  improved 
by  using  additional  (and  unconventional)  measurements  of  its  apparent  modes,  typically 
provided  by  an  imaging  sensor.  A  model  for  the  image-based  observation  channel  for  target 
mode  estimation  in  discrete  time  is  presented  in  this  paper.  A  multidimensional  point  process 
filter  is  obtained  by  making  use  of  the  discrete  time  point  process  theory  and  its  utilization  is 
illustrated  through  simulation  examples. 

10.  A  New  Approximation  for  the  Partially  Observed  Jump  Linear  Quadratic  Problem  (C.  Yang 
and  M.  Mariton,  to  appear  in  Int'l.  Journal  of  Systems  Science.  1991). 

We  consider  the  Jump  Linear  Quadratic  Problem  where  linear  state  dynamics  are  made 
contingent  upon  the  Markovian  transition  of  a  regime  variable.  It  is  desired  to  regulate  the 
state  while  minimizing  a  quadratic  performance  index.  In  the  case  of  partial  observations  the 
exact  solution  has  proved  to  be  elusive  and,  in  this  paper,  we  present  a  new  approximation 
based  on  the  optimal  solution  of  an  averaged  version  of  the  original  problem.' 


